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Abstract 

This essay explores the validity of including ELL students in British Columbia provincial high school exit exams. 
Data and a sample exam from the BC Ministry of education are used to scrutinize the practice of using BC 
provincial exams for high school exit decisions regarding ELLs. A comparison of failure rates of ELLs and 
mainstream students provides evidence for the inappropriateness of the BC provincial exams for ELLs. Ministry 
statistics reveal ELL failure rates ranging from twice to four times those of all students. An analysis of a sample 
exam found on the ministry website reveals confounding of content and language in the exam such that it is 
impossible to determine whether the exam is measuring language proficiency or content area knowledge. This 
exam shortcoming casts serious doubt on the legitimacy of using the provincial exam as an assessment for ELL 
learners. Possible negative consequences of including ELLs in these exams such as increased likelihood of 
school disappearance are discussed. Suggestions including providing alternative forms of assessment for ELLs 
are provided to facilitate managing the challenge of assessing ELLs attempting to obtain a high school diploma 
in BC. 

Keywords: high school exit exams, high-stakes tests, consequential validity, English language learners 

1. Introduction 

North American schools are becoming increasingly diverse (Goldenberg, 2006) which means that policy makers 
ignore at their peril the challenges that diverse learners face as they learn to cope with our educational systems. 
These demographic patterns are the same here in Canada. In fact, Statistics Canada reports that, by 2031, one 
quarter of Canadians will be immigrants from another country (Statistics Canada, 2010). 

Commenting on the American context, Menken (2008) makes a strong case that the era of No Child Left Behind 
and increased accountability has caused English language learners disproportionate academic hardship. In 
particular, she focuses on the devastating consequences that including ELL students in high-stakes standardized 
tests have had across the United States. She concluded that the cultural assumptions, linguistic complexity of 
these tests and the lack of sufficient accommodations explain why English language learners usually do not 
perform as well as native speakers (p. 96). The British Columbia high school provincial exams are unfair for 
English language learners (ELLs) because they were not constructed to assess ELLs yet high stakes decisions for 
ELLs are being made based on these exams. 

1.1 Background of High School Provincial Exams in BC 

Standardized testing is not a new phenomenon in British Columbia. Steffenhagen, (2008) reports that, according 
to Jerry Mussio, (formerly of the Ministry of Education and Canada's representative to PISA), provincial exams 
have a very long history in BC. In fact, provincial examinations of reading, writing and math were first 
introduced in 1876 and were regularly administered from 1925 until 1973. At that point, then Education Minister 
Eileen Daily cancelled standardized testing due to worries over the use of the tests. As part of Socred 
government’s efforts to “restore confidence” in public schools. Grade 12 exams were re-introduced in 1984. And 
they have remained in place ever since (Steffenhagen, 2008). At present, the debate in the province over the 
fairness and validity of these tests continues. 

In recent decades, debates over the appropriate uses of standardized test results have intensified as conceptions 
of test validity have changed quite dramatically. Messick’s (1989) introduction of the notion of consequential 
validity and the realization that validity is not an inherent trait of a test (Koretz, 2008) have compelled scholars 
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and test designers to rethink previously unquestioned assumptions about testing. Based on some of these insights, 
the issue to be addressed here becomes not about the validity of BC provincial exams per se but rather the 
validity of their use with ESL students and their possibly deleterious impact on these learners’ school 
trajectories. 

To earn a “Dogwood” diploma in British Columbia, all students must take provincial examinations that 
presumably cover the curriculum of grade 10 English, Science, and Math. They must also write grade 11 Social 
Studies (or BC First Nations Studies 12) and grade 12 English. The exam requirement as a prerequisite to 
earning a high school diploma makes them potentially high-stakes tests. That is, if students cannot pass these 
required tests, they will jeopardize their chances of obtaining the valued “dogwood” diploma. 

In addition to containing content and language that ELLs could not realistically be expected to know, these 
exams do not attempt to allow ELLs to display what they actually do know. Essentially these learners are 
bringing funds of knowledge (Moll, Amanti, & Neff, 1992) that are being ignored and undervalued by these tests. 
Corson (1998) points out “often culturally different children come to school knowing very different kinds of 
things from other children. When they meet the assessment methods that teachers use, they are sometimes asked 
to display knowledge about ‘x, y’ when what they really know about is ‘y, z’” (p. 169). That is, they are being 
punished for ignorance of things that they would not ordinarily be expected to know in their own cultural context. 
Shohamy (2000) notes that, in the past, test takers did not really have any rights. 

It was understood that there was specific knowledge that the test taker should have, but that the body of 
knowledge was defined by those who wrote the tests and that the test taker was expected to comply with their 
decision (p. 154). 

Based on the foregoing discussion, it could be argued that for these ELL BC provincial exam test takers not a lot 
has changed. One indicator of the inappropriateness of these tests for ESL students, they are only mentioned in 
the handbook of administration procedures to explicitly disallow them from using translating dictionaries. This 
reference to ELLs reveals two policies for testing these students. They are included in the exam yet they receive 
no accommodations (e.g., first-language versions or bilingual dictionaries) that might allow them to perform at 
their optimal level. 

Some might argue that it is appropriate to include ELLs in the provincial exams because they would question 
whether in fact these high school exit exams could be classified as high-stakes tests. Mathison (2004) defines 
high-stakes as “tests that have serious consequences attached to the results - these consequences may be for 
students, teachers, principals, schools and even states” (p. 122). This definition would seem to be appropriate for 
the BC provincial exams because if test takers do not pass the tests, they will jeopardize their chances of 
receiving their high school diploma. 

1.2 Previous Research into the Consequences of High School Exit Exams for ELL Learners 

The social costs of testing have received increasing attention from the scholarly community. Researchers are 
becoming progressively more aware of the negative social impact large-scale high-stakes tests can have on the 
lives of learners - particularly those who are most vulnerable - when the results of these tests are used to make 
decisions that unfairly limit the life choices of these learners. Shohamy (2007) asserts that “[w]e need to examine 
the ramifications of tests, their uses, misuses, ethicality, power, biases, and the discrimination and language 
realities they create for certain groups and for nations, and we need to use a critical language testing perspective 
[emphasis in original]” (p. 144-145). This is especially true with a test that serves to identify a learner as a high 
school graduate or not. 

Research into the use of high school exit exams with minority learners has accumulated some surprising findings 
which have led to some pessimistic conclusions. Darling-Hammond (2010) surveyed the research into 
high-stakes tests and minority students. She noticed a disturbing relationship between grade 9 retention rates and 
levels of performance on the grade 10 standardized tests. She reported that “many of the steepest increases in test 
scores occurred in schools with the highest retention and dropout rates” (p. 94). She cited a study by Wheelock 
which found that “some schools improved their scores and reaped [state] rewards by keeping students out of the 
testing pool or out of school entirely” (p. 94-95). Thus, it appears that schools raised their test scores but 
removing the weakest learners from the testing pool. This issue will be revisited in the discussion of test 
consequences for BC ELLs. 

Those in favor of continued testing of ELLs with the provincial exam might protest that the case of the BC 
provincial exam is somewhat different than many of the high-stakes state exams in the US. They might point out 
that BC test takers’ fate do not entirely depend on the results of the exam. In fact, in British Columbia, exam 
performance and grades in related class are weighted at 20% exam/ 80% course grade for those in grades 10-11 


2 




www.ccsenet.org/elt 


English Language Teaching 


Vol. 5, No. 9; 2012 


and 40% exam/ 60% course grade for those in grade 12. On the surface, this appears to be fairer for test takers 
except for the fact that those groups who fail the test at a higher rate than the mainstream population still stand a 
greater chance of not graduating, even if their grades are comparable. This potential disparity in results calls to 
mind Mathison’s (2004) observations that “the danger is that standardized test results will be weighed more 
heavily than they ought to be, that decisions once made cannot or will not be reversed, and that other compelling 
information may be ignored” (p. 122). In the case of BC, the provincial exams are not the only factor taken into 
account when deciding whether or not a student qualifies to graduate but this “snapshot” of ELL performance 
using a tool not designed for that purpose can have serious consequences for those who perform well enough in 
their coursework but do not pass the exam. This is particularly the case for the grade 12 Language Arts exam. 

2. Examination of Failure Rates and Exam Content 

2.1 ELL Failure Rates on British Columbia Provincial Exams over the Past 5 Years 

Shohamy (2000) was one early observer to raise the alarm about the effect that unfair testing practices could 
have on disadvantaged and vulnerable learners. She investigated the implementation of tests of Arabic and 
English in the Israeli context and concluded that “...test takers are the true victims of tests. They suffer badly 
from tests in this unequal power relationship between the test as an organization and the demands put on test 
takers” (p. 154). Similar patterns of the particularly damaging effects of using high-stakes tests with minority or 
disadvantaged learners have been reported in the American context as well. In her review of related research 
Mathison (2004) noted that: 

...there are clear patterns illustrating who is most likely to be subjected to these tests. High-stakes testing is 
disproportionately found in states with higher percentages of people of color and living in poverty’ ...of the 
eighteen states that require passing a standardized test to graduate, eleven are in the south, that is, states with 
substantial minority> populations (Mathison, 2004, p. 122). 

The issue here is the fairness of using the BC provincial exams to decide whether ELLs can graduate from high 
school in British Columbia. Two appropriate pieces of evidence to determine the validity of this practice would 
appear to be ELL test takers’ performance on the exams relative to their peers and the language and content of 
tests themselves. The following chart, taken from the BC Ministry of Education website, illustrates ELL 
performance for the examinable subjects over the past five years. 


Table 1. Percentages of Provincial Exam Failures for all Students and ESL Students* 


Exam 

2006/2007 

2007/2008 

2008/2009 

2009/2010 

2010/2011 

% of All Failing 
Students 

% of ESL Failing 
Students 

% of All Failing 
Students 

% of ESL Failing 
Students 

% of All Failing 
Students 

% of ESL Failing 
Students 

% of All Failing 
Students 

% of ESL Failing 
Students 

% of All Failing 
Students 

% of ESL Failing 
Students 

English 10 

7 

21 

8 

22 

8 

22 

9 

20 

10 

26 


(N = 

(N = 

(N = 

(N = 

(N = 

(N = 

(N = 

(N = 

(N = 

(N = 


3728) 

512) 

4078) 

575) 

4166) 

575) 

4515) 

528) 

5098) 

673) 

Essentials 

18 

33 

19 

43 

19 

29 

21 

49 

35 

MSK 

of 

(N = 

(N= 16) 

(N = 

(N = 26) 

(N = 

(N = 16) 

(N = 

(N = 29) 

(N = 67) 


Math 10* ** 

787) 


889) 


862) 


969) 




Science 10 

24 

28 

12 

13 

6 

9 

5 

9 

10 

15 


(N = 

(N = 

(N = 

(N = 

(N = 

(N = 

(N = 

(N = 

(N = 

(N = 


12244) 

717) 

6292) 

343) 

2818) 

237) 

2380) 

254) 

4910) 

397) 

Social 

7 

16 

4 

8 

8 

19 

6 

12 

9 

17 

studies 11 

(N = 

(N = 

(N = 

(N = 

(N = 

(N = 

(N = 

(N = 

(N — 

(N = 


3160) 

215) 

1924) 

130) 

3812) 

318) 

2936) 

200) 

4451) 

294) 

English 12 

7 

32 

7 

26 

10 

40 

9 

39 

7 

28 


(N = 

(N = 

(N = 

(N = 

(N = 

(N = 

(N = 

(N = 

(N = 

(N = 


3230) 

240) 

2889) 

200) 

4506) 

300) 

4181) 

277) 

3357) 

214) 

Note: 


* Data obtained from http://www.bced.gov.bc.ca/reports/pdfs/exams/req/prov.pdf 

** This is considered to be the intermediate level between “Applications of Math” which is geared more toward 
college and trades preparation and “Principles of Math” which is aimed toward preparing students for 
university-level study of math. 
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Figure 1. Comparison of percentage of failures on English 12 exam during 2006 through 2011 


Table 1 demonstrates that over the past five years across all required subject areas examined by BC provincial 
exams, ELL students have consistently had failure rates that are approximately double those of all students. 
These numbers in table 1 were calculated by subtracting the number of students who were reported to have 
passed the exam from the total number of students who took the exam during each academic year. The results for 
grade 12 English are a particularly dramatic example of the discrepancy in test scores (see figure 1). From the 
year 2006/2007 through to 2010/2011, ELL student failure rates were four times those of all students. Even for 
mathematics, a subject that is thought to require less English proficiency than others, ELL failure rates are 
approximately double those of all students. These statistics indicate that the practice of including ESL students in 
these exams must be called into question. 

Either ELL students are systematically receiving drastically inferior instruction or the test is not a valid 
measurement of their abilities. If it were, their failure rates would be equivalent to those designated as non-ELL. 
Given that ELL students consistently have almost double the number of failures (and almost quadruple in 
English 12) on average, one has to question the fairness of ELL students’ inclusion in these assessments. 

2.2 Bias as an Explanation for Discrepancies in BC Provincial Exam Failure Rates 

One possible explanation for the large discrepancies in failure rates is test bias. At present, there are no published 
residts of DIF analyses for the provincial exams that examine the performance of ELLs as compared with 
English speakers. Therefore, currently, the only option for identifying possibly biased items or content is to 
analyze exams to discover any content that might potentially cause ESL learners to perform differently than 
other test takers. 

An analysis of a sample grade 12 English exam provided on the Ministry of Education website demonstrated 
several specific test features that reveal the potential for these exams to be unsuitable for ESL learners for at least 
two reasons. First, the exams assume a great deal of cultural knowledge that ELLs could not be expected to have. 
The four year limit on an ESL designation in BC seems insufficient to acquire the nuanced and sophisticated 
cultural knowledge assumed by these exams. The grade 12 English (not literature, which is optional) exam found 
on the ministry website provides an illustration of the complex cultural knowledge required to perform well on 
the exam. One question worth 17% contains a reading passage about Andy Warhol’s first visit to Canada. To 
answer the questions, ELL learners must know about who Andy is and why it is ironic that Andy’s sculptures 
were considered taxable goods by Canadian customs officials. A second difficulty with the passage is that it is 
filled with colloquial words and expressions like “nada,” “deadpan comedy,” “a total dud” and “drop 80 works” 
among numerous others. Certainly the native speakers for whom test was designed should know these 
expressions but it is questionable whether designated ESL learners who were in the country for four years or less 
should. 

These test features demonstrate that these tests do not attempt to separate the confounded factors of content 
knowledge and language proficiency for ELL learners. Flowever, for these tests to be a considered the valid 
measure of content knowledge for ELL students that they purport to be - otherwise they should not be used with 
these learners - separating language and content knowledge is exactly what the exam must do (Solano-Flores & 
Trumbull, 2003). 

McNamara and Roever (2006) make an excellent point that traditional psychometric approaches to test 
validation have serious limitations in terms of their ability to comment on the social dimensions and 
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consequences of language tests. They note that “...researchers need to go beyond the test booklet to understand 
fairness and bias in its social context. After all, DIF analyses only detect bias, but the factors that cause 
advantages and disadvantages for groups of test takers and bias their educational opportunities lie in the larger 
social context” (p. 128). The ironic fact is that the social dimension of tests is what really matters the most. Tests 
treating learners differently is worthy of attention but tests that forever alter the course of test takers’ lives 
demand close scrutiny. 

3. Consequences of Failure for ELLs in British Columbia 

At this stage it is difficult to ascertain what the direct consequences of failing these exams are for ELLs in 
British Columbia. However, based on some relevant indicators some speculations can be made. Looking at the 
consequences of failure of exit exams in other contexts might provide some insight into what may be happening 
here. 

Heubert and Hauser (1999) observe that “ELLs are particularly vulnerable to high-stakes decisions based on test 
results; [for instance] tests used to determine high school graduation...” (cited in Menken, 2008, p. 97). Menken 
(2008) also highlights that 

[wjhile passing the Regents is challenging for anyone learning the English language, it is particularly so for 
older students who arrive in the United States during high school and students with limited literacy skills either 
in their native language or in English (p. 112). 

This situation is similar to what many ELLs in BC face. Older students come during their secondary years and 
they are unable to learn enough of the requisite academic language to pass the exams in the short time frame they 
have. As a result, they do not perform well on exams and their graduation status is placed in jeopardy. 

The most drastic consequence of high stakes tests noted by several scholars was the increased risk of students 
dropping out of school due to failure of a high-stakes high school exit exam. Indeed, it has been found that 
“...graduation tests...are having substantial impact on the numbers of students who are dropping out of school” 
(Mathison, 2004, p. 124). Darling-Hammond (2010) reported on the results of a large-scale summary of research 
conducted by The National Research Council. She noted their conclusion 

...that low-performing students who are held back because of their test scores do less well academically and are 
far likelier to drop out than comparably achieving students who are moved ahead in school with grade retention 
increasing the odds of dropping out by as much as 250%” (p. 76). 

This statistic certainly gives cause for alarm. It is even more disturbing that schools may even be encouraging 
low-performing students to leave. Menken (2008) mentions her great concern “that because of the challenges 
that Regent exams pose for ELLs and their high-stakes consequences, the exams act to push these students to 
leave school, either of their own volition or with their school’s encouragement” (p. 108). Certainly the original 
aim of these tests was to compel teachers to do more for at-risk students, but the outcome in some cases seems to 
be compelling them to do less. 

This relationship between failure of high-stakes and dropout rates in other contexts may also exist here in BC. 
The disappearance rates of English-as-an-additional-language (EAL) learners from schools across Canada 
(Toronto, 53%) (Radwanksi, 1987), (Calgary, 73%) (Watt & Roessingh, 2001) and in the Lower Mainland (60%, 
Gunderson & Murphy Odo, 2009) are disturbingly high especially when they are compared to the official 
provincial high school dropout rate of 6.2% (StatsCan, 2010). Of particular relevance here is the data from the 
Lower Mainland of BC. Gunderson (2007) conducted a large-scale study of immigrant students in the 
Vancouver area and observed a dramatic “disappearance” rate of ESL 54% of students from the district who 
could not be accounted for. It does not seem unreasonable to ask whether students might be disappearing from 
BC schools for similar reasons. 

Mathison (2004) made a startling observation that “students who have by other indicators done reasonably well 
in school are disproportionately influenced by poor test scores to drop out of school” (p. 127). That is, the 
students who often show the most promise at school are those who are the most negative affected by these tests. 
At this stage, it is difficult to determine whether this pattern is repeated in Canada. Gunderson’s (2007) 
investigation of ELL student achievement in Vancouver-area schools revealed that between grades 8 and 12, 
Canadian students had an average GPA of between 2 and 2.5. Likewise, ELL learners had an average GPA of 
between 2 and 2.5 though their GPA in 11 and 12 dipped below 2 in English and social studies. This gives some 
indication that, in terms of class grades in the examinable subjects, ESL students are not performing 
considerably differently than Canadian students. If that is the case, their failure rates on the provincial exams 
being twice as high as Canadians coupled with the high disappearance rates should be some cause for concern. 
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The potential connection between dropout rates and exit exams is disquieting but there are other more insidious 
consequences as well. Other examples of the deleterious effects that high-stakes tests have been found to have on 
ESL students include “... ELLs attending high school for more years, attending more classes per day, and 
attending more after-school and tutoring programs than native English speakers” (Menken, 2008, p. 105-106). 
Thus, it appears that in their struggle to persevere ELLs are having to work much harder than their mainstream 
peers. These observations overlook the fact that after-school tutoring is the luxury of those who are not occupied 
with part-time jobs to help the family make ends meet. In this way, those ELLs who are disadvantaged in terms 
of social and economic capital arguably suffer even more disproportionately when compared to ELLs who have 
access to more resources. 

Of course the struggles that ELL students face with these tests are complex. They are compounded by the fact 
that the label ELL itself, though at times useful, can also mask the widely diverse needs and access to economic 
and social capital that its members have. On the one hand, many ELLs struggle to acquire the language they need 
to do well at school but there are often vast differences in the amount and kinds of support to which they have 
access. In his study of ELLs in the Vancouver area, Gunderson (2007) noted that while many ELLs struggled in 
school there were also those from affluent backgrounds who were able to manage with these exams because they 
are able to hire after-school tutors to help them. This phenomenon underscores the wide diversity within the ESL 
label. 

A final potentially dangerous effect of these high-stakes tests is on ELLs’ identities. The core of who they are. 
Foucault argues that tests can shape our identity in powerful ways. Tests have the power to tell us who we are 
and we often believe them. McNamara and Roever (2006) contend that “...tests become technologies of 
subjectivity. They act as mechanisms both for the definition of subject positions and for the recognition of 
subjects. Tests create the identities they measure” (p. 196). This observation is particularly important when 
considering high school exit tests. If tests create the identities they measure then failing a test that purports to 
measure the identity of high school educated English user sends the message that the test taker is not suited for 
secondary - or post-secondary - education. When ESL learners are defined, and come to define themselves, as 
deficient there must certainly be negative psychological effects. Based on the confluence of findings in her 
research, Menken (2008) reported that there are. She states “a primary theme that repeatedly emerges in 
interview data with administrators, teachers and students is that high-stakes tests place tremendous pressure on 
ELLs and failing the exams negatively impacts their self esteem” (p. 113-114). This crippling stress and lowered 
self esteem are very likely the main contributors to many ELLs’ decision to leave school without completing. 

4. Addressing These Issues with BC Provincial Exams 

Opponents might attempt to argue that if the system is so unfair it is the responsibility of the test takers or their 
advocates to do something to change it. However, as Shohamy (2000) explains, speaking out against flawed and 
unfair testing practices 

...requires a lot of courage on the part of test takers to battle powerful testing organizations, especially in a 
high-stake test where results are so detrimental. This explains why individual test takers rarely appeal. There is 
a feeling on the part of individuals that there is just too much to lose (p. 155). 

Additionally, many immigrants are not knowledgeable of the North American education system so they do not 
know how or where to go to express their displeasure about the current situation (Louie, 2001). Similarly, they 
may also lack the social or political capital to actually effect some change if they do complain (Louie, 2001). 

At present, the best hope appears to be for concerned stakeholders and community members to engage in public 
debate about the fairness of these assessments for vulnerable or disadvantaged groups. Solutions have been 
proposed to solve similar problems in other contexts. Reviewing some of these ideas may provide useful 
guidance for how these issues might be addressed locally. 

Probably the most practicable adjustment to make these tests more valid for ESL learners is to not include ELL 
learners in this assessment and to design a separate test that more accurately reflects their abilities (Solano-Flores 
and Truball, 2003). This more resource intensive option could also be an adaptation to the existing test to better 
meet the needs of ESL students. Possible test versions could be a locally-developed assessment that takes ESL 
learners’ culture and language into account or a dual-language test. Of course, the feasibility of this practice is 
questionable because, to be equitable, developers would need dual-language assessments for all language groups. 
If those responsible for using the tests lack resources to design exams that accurately measure ELL students’ 
capabilities, they should be prepared to conduct research to identify appropriate accommodations (e.g., the use of 
dictionaries or more time) to ensure as fair an assessment as possible. Access to suitable accommodations may 
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help ELLs better demonstrate their knowledge while allowing the testing process to remain economically 
feasible. 

Another option would be to rely on alternative assessments. These might be based on some type of portfolio 
assessment that allows students to demonstrate their areas of strength. A final option might be to weight the 
results of provincial exams and class grades differently for ESL or other potentially-disadvantaged groups. As 
was noted above, it appears that ESL students do not necessarily achieve vastly differently than their mainstream 
peers in terms of the grades that they receive in their classes. Providing a greater weighting for their class grade 
may give students who do poorly on the test a chance to offset their test score with their better class grade. This 
would allow them to pass and graduate. 

More research into the consequences of these tests for these learners is needed. In particular, the ultimate fate of 
ESL learners who have difficulty passing these tests needs to be learned to ensure that they are not falling 
through the cracks and that the promise of the better life they came to Canada to pursue does not turn out to be 
empty. Policymakers may cling to the misguided notion that including everyone in these tests is somehow fair. 
However, we should all bear in mind the old adage that fairness is not treating everyone the same, it is giving 
everyone what they need to thrive. We need to learn what these students need to thrive and give them that. 
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